[IO] Adding functions to estimate max byte size of all fields in a pool #1217
Conversation
src/framework/mpas_stream_manager.F
Outdated
| character :: tmp_char | ||
| integer(kind=I8KIND) :: max_field_bytes, field_bytes | ||
| integer(kind=I8KIND), parameter :: int_size = c_sizeof(tmp_int) | ||
| integer(kind=I8KIND), parameter :: real_size = c_sizeof(tmp_real) |
There was a problem hiding this comment.
It may be possible to avoid creating temporary variables only to check their size. I think it might work, for example, to write
integer(kind=I8KIND), parameter :: real_size = c_sizeof(1.0_RKIND)There was a problem hiding this comment.
This seems to work for RKIND, but I wasn't sure about how it works for character and int. Got some build errors.
| call mpas_pool_get_field(allFields, itr % memberName, real5d, timeLevel) | ||
| field_size = global_dim_size(real5d % block, real5d % dimNames, real5d % dimSizes, info % nDims) | ||
| end select | ||
| field_bytes = int(field_size, kind=I8KIND) * real_size |
There was a problem hiding this comment.
Perhaps field_size should be declared as an I8KIND, and this type conversion should be eliminated.
There was a problem hiding this comment.
Isn't there a type conversion involved whether it's in this function or inside global_dim_size . Not sure if mpas_pool_get_dimension can work with I8KIND int pointers?
There was a problem hiding this comment.
I've now changed this. The only type conversion happens in global_dim_size , due to mpas_pool_get_dimension forcing a regular int. And I've changed to mpas_dmpar_sum_int8 now
abishekg7
changed the title
This PR introduces two functions to
src/framework/mpas_stream_manager.Fto help compute the max byte size given a field pool from an output stream. This information can be used to decide whether to use CDF2 or CDF5 as the output file format. See #1219function
stream_max_var_sizeaccepts a field pool and iterates through each field callingglobal_dim_sizeto obtain the dimension size of the current field over the entire domain, then computes the number of bytes occupied and the max bytes size over all the fields in the poolfunction
global_dim_sizeiterates over all the dimensions in a given field to compute the overall field size, obtaining the global dimension size if the dimension is decomposed. IfisVarArray=.true., it skips over the left-most dim, which tracks the size of the variable list, and is not present in the output file.